Network-Extensible Reconfigurable Media Appliance

ABSTRACT

Extensible reconfigurable media appliance for security and entertainment captures images digitally for storage. Digital effects and filters are applied to incoming video stream on-the-fly or to video data stored in memory. Digital effects and filters are dynamically stored, modified, updated or deleted, providing extensible reconfigurable effects studio. Digital media appliance communicates wirelessly with other media appliances, computers, security systems, video storage, email, chat, cellular services or PDAs to provide seamless integration of captured video stream.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a divisional of U.S. patent application Ser. No.11/203,645 filed on Aug. 11, 2005, which is a continuation of the U.S.patent application Ser. No. 10/448,202 filed on May 28, 2003.

BACKGROUND

1. Field

Invention relates to electronic media devices and processing methods,particularly to networked reconfigurable media appliance.

2. Related Art

Conventional digital video media appliances rely on external computingresources for post-processing of recorded audio or video signals, sincelimited on-appliance editing and filtering capabilities may render suchtasks impractical. Furthermore, limited on-appliance softwareextensibility and limited communication with external resources leaveconventional digital media appliances as standalone audio or videorecording tools limited by factory-loaded on-appliance processingcapacity.

Accordingly, there is need for network-extensible and easilyreconfigurable media appliance capable of communicating over networksand allowing for extension of on-appliance audio or video processingsoftware and tagging of recorded audio or video signals.

SUMMARY

Network-extensible reconfigurable media appliance senses incoming audioand/or video, and encodes and stores in media appliance memory oralternatively relays over network to recipient. On-appliance digitalaudio and/or video effects and filters process audio and/or video datastored in memory, as well as incoming audio and/or video streamon-the-fly. Media appliance dynamically stores, modifies, updates anddeletes on-appliance set of digital effects and filters, providingmobile extensible reconfigurable effects studio. Media appliancecommunicates wirelessly and/or over-wire with other media appliances,computers, security systems, video storage, Global Positioning System(GPS) services, Internet, cellular services and/or personal digitalassistants (PDA) providing seamless integration of captured audio and/orvideo stream with off-appliance resources and/or services. Audio and/orvideo, wireless, biometric and GPS input and/or output as well ason-appliance acceleration detector render media appliance suitable forsecurity applications. Extensible reconfigurable on-appliance effectsand filters studio render media appliance for entertainment and videoproduction or editing applications.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 a is an architectural diagram illustrating network-extensiblereconfigurable media appliance according to an embodiment of the presentinvention.

FIG. 1 b is an architectural diagram illustrating network-extensiblereconfigurable media appliance according to an embodiment of the presentinvention.

FIG. 2 is a diagram illustrating memory element of network-extensiblereconfigurable media appliance according to an embodiment of the presentinvention.

FIG. 3 a is a diagram illustrating network-extensible reconfigurablemedia appliance communicating with other networked services and devices,according to an embodiment of the present invention.

FIG. 3 b is a diagram illustrating network-extensible reconfigurablemedia appliances communicating over a network with a server, accordingto an embodiment of the present invention.

FIG. 4 is a flow diagram illustrating a method for sensing according toan embodiment of the present invention.

FIG. 5 is a flow diagram illustrating a method for filling in a templateaccording to an embodiment of the present invention.

FIG. 6 is a flow diagram illustrating a method for tagging audio and/orvideo representation with location and/or direction information.

FIG. 7 is a flow diagram illustrating a method for transferring dataand/or instructions from off-appliance source to on-appliance memory.

DETAILED DESCRIPTION

FIG. 1 a is an architectural diagram illustrating network-extensiblereconfigurable media appliance 100 according to an embodiment of thepresent invention. Media appliance 100 comprises media unit 101,controller 108, communication unit 103, and power module 116.

Media unit 101 comprises audio and/or video (A/V) sensor 120 for sensingincoming audio and/or video. Sensed video is stored in memory 110 usingvideo format such as Digital Video Disc (DVD), PAL Digital VideoCassette (PAL DVC), PAL or NTSC Laserdisc, 24P HD, ¾-inch, MPEG-2,MPEG-4 (DV-25, DV-50, IMIX, ISMA, etc.), H.264, AVI, DV, DVCAM, DVCPRO,DVCPRO-25/50/100, VHS, D-VHS, W-VHS, Digital-8, Digital-S, D1, D2, D5HD, Betacam SX, Digital Betacam, other digital ENG format, Motion JPEG,any other HDTV format, NTSC, PAL, HDD/RAID/Disk Arrays, and/or otherformat for encoding video (specifications for describing these formatsare herein incorporated by reference).

Media unit 101 optionally comprises biometric module 106. Biometricmodule 106 comprises finger-print scanner, retinal scanner, and/or otherelement for collecting a biometric sample, and stores scanned biometricdata and/or result of biometric identification process in memory 110.For example, a data structure is stored comprising a digitalrepresentation of collected biometric sample for authorization based oncomparison with previously-stored biometric identifier. Biometric module106 optionally couples with a micro-array chip for genetically-basedidentification.

Media unit 106 optionally comprises reconfigurable logic and/or software122 for performing programmable audio and/or video sensing, or forconversion to or from audio and/or video formats.

Controller 108 comprises microprocessor 123 (such as one from the IntelCentrino processor family, the specification of which is hereinincorporated by reference), and optionally comprises co-processor 124,Digital Signal Processing (DSP) unit 125, array processor 126, and/orreconfigurable logic 127. Controller 108 performs audio and/or videoprocessing on audio and/or video data residing in memory 110. Optionallyin real-time manner, controller 108 performs on-the-fly audio processingand/or on-the-fly video processing on incoming encoded audio data and/orincoming encoded video data prior to storage of resulting processedaudio data and/or resulting processed video data in memory 110.

Controller 108 is implemented in Application Specific Integrated Circuit(ASIC) blocks, synthesizable intellectual-property cores, cellprocessors, reconfigurable logic blocks, Field Programmable Gate Arrays(FPGAs), Tensilica's XTensa chip architecture and/or instruction set,Single or Multiple Instruction Single or Multiple Data (S/MIS/MD)architecture signal processing chips, Sony “Cell” chip, and/or otherarchitecture for performing audio and/or video processing.

Controller 108 and/or A/V sensor 120 may perform color-space conversion,brightness, white-balance, backlight compensation, gain control,activity detection, motion detection, motion tracking, gamma correction,sharpening, multi-frame noise reduction, depth estimation, 2-D bad-pixelcorrection, video compression, video stabilization, digital pan, digitaltilt, digital zoom, and/or mosaicing for building panoramic images fromsuccessive frames.

Communication unit 103 comprises radio-frequency (RF) transceiver 128for communicating via radio waves (e.g. over cellular or other wirelessnetwork), and/or network controller 129 for communicating via a wiredand/or wireless network (e.g. local area network (LAN), wide areanetwork (WAN), wireless fidelity (WiFi) network, etc.).

Communication unit 103 optionally comprises subscriber informationmodule (SIM) unit 130 and/or smart card unit for storage and/orretrieval of information about a user (such as user preference,subscribed service, permission, account information, etc.), and/or forallowing usage of media appliance 100 by one or more users.

Communication unit 103 optionally comprises GPS module 112 for receivingGPS data over satellite. Optionally, GPS module 112 is a micro GPStransponder implemented in single chip or chipset.

Communication unit 103 optionally comprises acceleration detector 113(such as a gyroscope, a single-chip accelerometer or other element fordetecting acceleration) for determining orientation and/or accelerationof media appliance 100.

Communication unit 103 optionally comprises reconfigurable logic orsoftware 131 for performing programmable protocol translation, formatconversion, network packet processing, network packet compression and/ordecompression, communication encryption and/or decryption, and/or othercommunication processing.

Power module 116 provides power for media appliance 100, and comprisesAC and/or DC source, portable rechargeable battery, fuel cell (e.g.direct methanol fuel cell, etc.), and/or other source for providingelectrical power. Optionally, media appliance 100 employs MICAmicrosensor platform for low-power wireless sensor networks, hereinincorporated by reference.

Optionally, media appliance 100 architecture conforms to AdvancedTelecommunication Computing Architecture (AdvancedTCA), hereinincorporated by reference.

FIG. 1 b is a diagram illustrating network-extensible reconfigurablemedia appliance 100 according to one embodiment of the presentinvention. Light or video sensor 102 senses incoming image stream andstores digital representation in memory 110. Preferably, sensor 102 is acomplementary metal oxide semiconductor (CMOS) image sensor. Optionally,sensor 102 is integrated with an image preprocessor. Optionally, sensor102 comprises integrated two-chip set such as Pixim D1000 or D2000 VideoImaging System chip sets. Sensor 102 optionally comprises a partitionfor post image processing steps. Alternatively, sensor 102 is acharge-coupled device (CCD) or an Active Pixel Sensor (APS) imager.Audio sensor 104 senses incoming sound and stores digital representationof incoming sound in memory 110 using audio format such as AudioInterchange File Format (AIFF), MPEG Layer 3 (MP3), and/or other formatfor encoding audio information.

I/O module 111 preferably has audio and video outputs. I/O module 111preferably communicates with on-appliance display or screen unit 114 andon-appliance speaker 115 for displaying video and generating audio.Optionally, display unit 114 comprises a teleprompter for displayingvisual prompts (such as text and/or pictures).

Optionally, I/O module 111 communicates wirelessly, wired, over cellularnetwork, over LAN and/or over WAN (such as Internet), to send and/orreceive GPS data, Digital Rights Management (DRM) meta-data, audioand/or video plugins, and/or other instructions and/or data forprocessing and/or tagging of audio and/or video data. Optionally, I/Omodule 111 has video and audio inputs for receiving audio and videosignals from external audio and/or video source such as a camera, a PDA,a media repository, a satellite, a security service, a DRM service, abiometric service, a GPS service, a PC or workstation, a cellularservice or cellular device, or other device or service communicatingwith media appliance 100. Media appliance 100 optionally has networkcontroller 117 for communicating with other devices and/or services overa network.

FIG. 2 shows memory 110 according to a preferred embodiment of thepresent invention. Memory 110 comprises Dynamic Random-Access Memory(DRAM), Static Random-Access Memory (SRAM), high-speed Flash memory,and/or removable memory (e.g. removable flash memory card such asMultiMediaCard). Memory 110 stores audio and video data 201.

Optionally, memory 110 stores software instructions and dataimplementing billing 202 and/or business methods, such as a time-basedpay-per-view and/or micro-billing feature. For example, memory 110stores a data structure comprising a field describing a viewing (such asa home-viewing of a video clip of video stream) and/or a fieldindicating an amount to be charged for the viewing and/or a fieldidentifying a party to be charged.

Optionally, memory 110 stores meta-data and/or instructions forimplementing DRM 203 (e.g. Disney Media Asset Management (MAM) format),Resource Definition Framework (RDF) implementation such as Adobe's XMP(eXtensible Metadata Framework), or other scheme for managing meta-data.For example, an XMP packet data structure comprising a header, an XMLmeta-data, a trailer, and a padding field is employed. Optionally,memory 110 stores data and/or instructions for implementing DRMaccording to a Right Expression Language Data Model, for exampleemploying Extensible Rights Markup Language (XrML). Optionally, memory110 stores meta-data and/or instructions for implementing proposedGlobal Release Identifier Syntax (GRID), for example employing a datastructure having an Identifier Scheme, an Issuer Code, a Release Number,and a Checksum.

Optionally, memory 110 stores instructions and/or data 204 forperforming digital authentication, encryption, decryption, keygeneration, digital signing, digital watermarking, and/or otherinstructions for performing security and/or privacy related computationon audio and/or video data, DRM data, billing data and/or conditions,sensitive personal data, or other data residing in media appliance 100and/or communicated to or from media appliance 100. For example, memory110 stores a data structure comprising a field describing an encryption(and/or decryption) key, and further stores instructions for encryptinga video stream using the encryption (and/or decryption) key.

Optionally, memory 110 stores instructions and/or data 205 forperforming identity recognition (such as facial recognition, emotionrecognition, voice recognition, and/or other pattern or identityrecognition) on video data 201 and/or on incoming video signal. Forexample, memory 110 stores a data structure comprising an identifier fora database against which image recognition is to be performed, forexample a database of faces for recognizing faces in a crowd. Thedatabase may be stored (partially or completely) internally on mediaappliance 100 or reside externally on a server. As another example,memory 110 stores a data structure comprising a feature extracted from avideo stream and/or video clip (using image extraction instructionsstored in memory 110), and the extracted feature is used for a data basequery or is sent to a server for further handling.

Optionally, memory 110 stores instructions and/or data for performingauthoring 206 and/or digital video editing (e.g. linear or non-linearediting), compositing, and/or special effects, such as Apple's Final CutPro software. For example, memory 110 stores a data structure comprisinga bit rate associated with the encoding of a video clip and/or videostream. As another example, memory 110 stores a data structurecomprising author information, genre information, title, characters,actors, genre, story, activities, viewer demographics, locations,scenes, backgrounds, props, objects, set pieces, or other informationpertaining to a video clip and/or video stream.

Optionally, memory 110 stores instructions and/or data for tagging 207the digital representation of a sensed scene (video stream and/or videoclip) with meta-data. For example, memory 110 stores a data structurecomprising time, media appliance location (such as provided by GPSmodule 112), media appliance orientation and/or media applianceacceleration (such as provided by acceleration detector 113),multi-lingual features (allowing for translation, subtitles, voice-over,etc.), cues to a theater automation system (such as instructions forhouse lights to go up, half-way up, or down, or instructions to open orclose curtains, etc.), instructions for allowing or disallowing content(such as trailers or promotional clips) to play next to other similarcontent, information indicating suitability of content for differentaudiences such as children, information indicating any promotionaloffers, products and/or services (such as advertisements, productcatalogs and/or coupons for products and/or services), informationallowing for organizing and/or managing meta-data available toadvertisers and/or service providers, and/or other informationdescribing, identifying and/or relating to content.

DRM meta-data and/or instructions optionally comprise flags forimplementing rights and/or limitations of reproduction, rights and/orlimitations of public performance, rights and/or limitations of display,rights and/or limitations of distribution, rights and/or limitations ofimportation, rights and/or limitations of transmission or access, rightsand/or provisions under Digital Millennium Copyright Act (DMCA), rightsand/or limitations of caching, rights and/or limitations of browsing,rights and/or limitations of storage, rights and/or limitations oftransfer such as burning to Compact Disk (CD) or DVD, rights and/orlimitations of referring or linking or framing, rights and/orlimitations of streaming or downloading, rights and/or limitations ofadvertising, or other rights and/or limitations and/or provisions. Forexample, memory 110 stores a data structure comprising a fieldidentifying a video clip or video stream, and a field for indicatingwhether a reproduction right is granted for the identified video clip ofvideo stream. In another example, memory 110 stores a data structurecomprising a field identifying a video clip or video stream, and a fieldfor indicating whether a public performance (and/or display) right isgranted for the identified video clip of video stream. Other digitalrights can be implemented analogously. DRM meta-data and/or instructionsoptionally support secure promotion, sale, delivery, distribution,and/or usage tracking of digital content. Optionally, executionenvironment is partitioned into kernel versus user space and/or intostandard versus trusted partitions according to Microsoft'sNext-Generation Secure Computing Base (NGSCB).

Media appliance 100 optionally inserts, deletes, and/or modifies a labelin an RDF (e.g. XMP) tag describing a media segment.

Media appliance 100 optionally implements content authenticity, deviceauthentication, and/or user authentication. Content authenticitycomprises digital watermarking, digital fingerprinting, and/or othertechnique for content authentication. For example, memory 110 storesinstructions for reading an identifier describing a source of a videoclip and/or video stream, wherein the identifier is embedded in adigital watermark within the video clip and/or video stream. As anotherexample, memory 110 stores a data structure comprising a fieldidentifying one or more authorized sources for downloading video clipsand/or video streams. Device authentication comprises smartcards, publickey certificates, and/or device for performing authentication. Userauthentication comprises biometrics using biometric module 106,passwords, and/or other technique for performing user authentication.

Media appliance 100 optionally implements, in software (e.g. residing inmemory 110) and/or hardware, an abstraction layer between applicationand display, such as DVB (Digital Video Broadcast) and/or MHP(Multimedia Home Platform) abstraction layers. Specifications forincorporating the DVB and MHP formats are herein incorporated byreference.

FIG. 3 a shows networked media appliance 100 communicating with otherdevice and/or service, according to a preferred embodiment of thepresent invention. Communication with other device and/or serviceproceeds via direct network connection, Internet, WiFi, IEEE 802.11,IEEE 802.16, IEEE 802.15.4, ZigBee specification, cellular, Bluetooth,Universal Serial Bus (USB), Apple's FireWire, and/or other communicationchannel or protocol. Communication is optionally encrypted,authenticated and/or digitally signed, preferably with encryption engine204 implemented in memory 110, or alternatively with encryption engine204 implemented in controller 108.

Media appliance 100 optionally communicates with media repository 307for downloading and/or uploading video and/or audio clips, video and/oraudio meta-data such as author information, genre information, title,characters, actors, genre, story, activities, demographics, locations,scenes, backgrounds, props, objects, set pieces, etc.

Media appliance 100 optionally communicates with DRM service 308 fordownloading and/or uploading DRM meta-data. Optionally, media appliance100 generates a message indicating an infringement and/or otherviolation of digital rights, according to a set of DRM rules, such ascopying without permission, broadcasting without permission, etc. Forexample, memory stores a data structure comprising a field identifying avideo clip and/or video stream, and an indicator of a violation of a DRMrule, such as an act of broadcasting the video clip and/or video streamwithout permission.

Media appliance 100 optionally communicates with security service 309 toupload security information such as video and/or audio record of scene,identity recognition data as computed by identity recognitioninstructions 203, GPS data as provided by GPS module 112, directionaldata as provided by acceleration detector 113, and/or to downloadsecurity information such as location to watch, identity data to storefor matching against images, and/or voice audio signature to store formatching against audio clips. For example, media appliance 100 sends adata structure to security service 309, wherein the data structurecomprises a field identifying a person, and a field identifying thelocation of the media appliance 100 at the time the person is sensed bymedia appliance 100. Optionally, media appliance 100 couples to policeauthority for providing live and/or recorded footage and/or triggeringalarm and calling police according to built-in media applianceintelligence for identifying potential dangerous and/or suspiciousconditions.

Media appliance 100 optionally communicates with biometric service 301to upload biometric information obtained by biometric module 106, and/orto download biometric signature for matching against incoming biometricdata.

Media appliance 100 optionally communicates with GPS service 302, suchas GPS satellites, to receive GPS information. For example, if mediaappliance 100 moves into a restricted area, as indicated by GPS service302 and/or by information residing on media appliance 100 and/orobtained remotely, GPS unit 112 activates an alert. For example, memory110 stores a data structure comprising a field identifying a restrictedgeographical area, and media appliance 100 generates an alarm whenlocation of media appliance 100, as indicated by GPS service 302, fallswithin the restricted geographic area.

Media appliance 100 optionally communicates with news service 310 and/orother objective information service. In one embodiment, media appliance100 receives a data structure from news service 310, the data structurerepresenting a digital template and comprising a field identifying alocation, and one or more fields identifying elements to be covered byreporter (such as a person to interview, a particular place to point outto viewers, other news reporters covering the same news story, etc.).

Media appliance 100 optionally communicates with sports broadcastingnetwork, game-show broadcasting network, and/or other gaming orcompetition-related network 311. In one embodiment, media appliance 100receives a data structure from sports broadcasting network 310, the datastructure comprising a field identifying one or more competing parties,a field identifying a location of the competition, and a fieldindicating the competition schedule.

Media appliance 100 optionally communicates with private service 312. Inone embodiment, media appliance 100 receives a data structure from movieproduction source or network 310, the data structure comprising a fieldidentifying one or more movie or media production, a field identifying alocation of the production, a field indicating the production schedule,a field indicating one or more scenes, and a field indicating one ormore cast or staff members.

Media appliance 100 optionally communicates with renderer 313 to displayvideo data. Renderer 313 comprises a cinema or movie theater, televisionreceiver, computer display, IMAX display, a Digital Audio Broadcast(DAB) broadcaster, a satellite broadcaster, a digital TV, a HighDefinition TV (HDTV), a PDA and/or cellular phone (or other mobiledevice display).

Media appliance 100 optionally communicates with a personal computer(PC) and/or workstation 303 and/or other computing device forsynchronization of data residing on media appliance 100 with computer303 (optionally interfacing with media repository manager and/or programmanager residing on computer 303). For example, memory 110 stores a datastructure comprising a field indicating the time of last synchronizationof media appliance 100 with computer 303 (or media repository manager orprogram manager residing on computer 303). Communication proceedswirelessly and/or via a cradle (coupled to computer 303) into whichmedia appliance 100 is placed for synchronization. In one embodiment,media appliance 100 comprises a user interface offering asynchronization button (hard button on media appliance 100 and/or softbutton displayed in media appliance's 100 graphical display), activationof which causes described data synchronization.

Media appliance 100 optionally communicates with PDA 304, cellularservice and/or device 305, and/or other mobile service and/or device fordisplaying video and/or audio data.

Media appliance 100 optionally communicates with other networked mediaappliance 306 for exchanging video and/or audio clips and/or forcollaborating in the production of a media project, wherein a mediaappliance is assigned a token (number, string, etc.), statically ordynamically, for identifying the media appliance. Media appliance 100optionally communicates with other networked media appliance 306 toenable video-conferencing and/or multi-way collaboration, for example,in business meetings, real estate transactions, distance learning,sports, fashion shows, surveillance, training, games, tourism, etc. Forexample, memory 110 stores a data structure comprising a field fordescribing a group of collaborating media appliances 100, and a fieldidentifying media appliance 100 itself among the group of collaboratingmedia appliances.

FIG. 3 b is a diagram illustrating network-extensible reconfigurablemedia appliances communicating over a network with a server, accordingto an embodiment of the present invention. One or more client mediaappliances 330 communicate over a network 331 with server 332. Network331 is a combination of one or more wired and/or wireless networks suchas the Internet, a LAN, a WAN, a satellite network, or other network forcommunication. In one embodiment, server 332 is a news server, having ascript or digital template for producing a news program. Server 332delegates the recording or streaming of various predetermined pieces ofaudio and/or video footage to the various media appliance clients 330,wherein the recorded or streamed pieces will serve to fill-in the server332 script or digital template for producing the news program. Inanother embodiment, server 332 is a server for sports or othercompetition, having a script or digital template for producing a sportsprogram or a program for other competitive activity. Server 332delegates the recording or streaming of various predetermined pieces ofaudio and/or video footage to the various media appliance clients 330,wherein the recorded or streamed pieces serve to fill-in the server 332script or digital template for producing the sports (or othercompetition) program.

In one embodiment, I/O module 111 presents a user interface (UI),comprising a combination of hard (physical) buttons and/or soft(graphical) buttons for accessing and using billing functions, DRMfunctions, authentication, identity recognition, digital editing ofmedia, and/or other services as shown in FIG. 3 a and described above.For example, a view (for example comprising a button) is presented viadisplay 114 to allow approval of a billing associated with the viewingof video data. As another example, a view is presented via display 114,allowing selection of one or more audio and/or video data for submissionor transmission to a server 332, such as a news server or a sportsserver, as described above. Selection of a presented audio and/or videodata designates the selected data for submission or transmission to theserver. Optionally, interfaces and media appliances are physicallyseparate, wherein through an interface a user can tap into a pool or oneor more media appliances to view available audio and/or video data,and/or select one or more available audio and/or video for submission ortransmission to a server 332, as described above. As another example, aview is presented at server 332 for approving the inclusion of asubmitted or transmitted audio and/or video data into a script or adigital template for a news or sports program, wherein the audio and/orvideo data is submitted by a media appliance client 330 to server 332,as described above.

FIG. 4 is a flow diagram illustrating a method for sensing according toone embodiment of the present invention. The method begins withpre-production 401. Pre-production comprises employing 402 a scriptand/or storyboard flowchart, or employing 403 a digital template 403. Aportion of this front-end may be implemented automatically or manuallyin software, comprising analysis, design, development, production,implementation or evaluation of script, storyboard, and/or digitaltemplate. Optionally, frames and/or scenes are labeled (via meta-data)according to script, storyboard, or digital template in use.

A script or storyboard is downloaded over a wired and/or wirelessnetwork, made available via removable storage (e.g. memory card and/ordisk), or is alternatively created on media appliance. A digitaltemplate describes how to construct a video and/or multimedia documentby sensing (i.e. “shooting” or recording) and assembling individualscenes and/or segments in particular order, and is downloaded over awired and/or wireless network or created on media appliance.Alternatively, user of media appliance 100 may decide not to consult ascript, storyboard, or digital template, and proceed directly to sensing404.

One example of a template is a template for insurance inspection ofvehicle accidents, wherein the template indicates “slots” for videoclips, taken from various angles, of the vehicles involved in theaccident, as prescribed by an insurance company.

Optionally, media appliance 100 adaptively guides media applianceoperator in making discretionary decisions to take alternate scriptpaths and/or alter flow of script (or storyboard or digital template) orgenerally deviate from the script, for example when dealing withemergency conditions and/or events which do not occur according toscript. Such guidance may employ non-deterministic scripts, according tologic specified using Bayesian modeling, neural networks, fuzzy logic,and/or other technique for making decisions under complex conditionsand/or under incomplete information. For example, in one embodiment acast member in a script is described by fuzzy attributes, such as “afemale actor with at least five years drama experience” in leading role(instead of or in addition of identifying the lead role actor by name).Then, in case the lead actor canceling her engagement, instructionsemploying fuzzy logic perform a search for actors matching the fuzzyattributes to dynamically recommend one or more candidates to fill therole.

Optionally, digital template or script is non-linear, allowing for oneor more branching points. A branching point allows the script and/ortemplate to flow in more than one path. For example, scene (or clip orstream) A can be followed by scene B or scene C, depending on whichbranch of the branching point following A is taken. For a viewer, amedia presentation prepared according to such non-linear template orscript allows for a multiplicity of presentations comprising differentscene (or clip or stream) orderings. For a viewer, the decision of whichof the alternate paths to follow in a branching point can be viewerselected, randomly chosen, based on external variable (such as acombination of one or more of: weather, temperature, stock quotes, timeof day or year, viewing location, amount of money left in viewer'saccount, or any other external variables), based on biometric sensing ofviewer, based on the result of an identity or emotion recognitionprocedure on viewer (such as distinguishing between happiness, sadness,excitement, apathy, interest in a particular aspect of the presentationand/or other emotions or indications of interest exhibited by viewer),based on real-time input from viewer or from larger audience (such asdeliberate viewer decision of which script or template path to takenext, provided via an input device or detected by the presentationmodule), or based on other variables. Such non-linear template or scriptallows for example for the production and presentation of a PG-rated,R-rated, or X-rated version of a given movie depending on the audience(for example a parent may elect to view the R-rated version of the moviewhile electing a PG-rated presentation for the children). As anotherexample, a wedding template or script may allow for differentpresentations based on whether the bride's family or the groom's familyis viewing. As another example, a mystery presentation may offeralternate endings, based on viewer input or external variables asdescribed above.

Media appliance 100 senses 404 video and/or audio and stores a digitalrepresentation in memory 110. Optionally, multiple audio and/or videostreams are sensed, either by the same media appliance or bycollaborating media appliances, wherein synchronization is provided forthe multiple streams, in the form of meta-data tags describing relatedscenes and/or streams and/or frames, and/or in the form of meta-datadescribing time stamps relating different scenes and/or streams. Forexample, memory 110 stores a data structure comprising one or morefields identifying one or more related video scenes and/or streamsand/or frames, and a field indicating the nature of the relation (forexample indicating that the video scenes and/or streams and/or framesrepresented different viewing angles of the same sensed object).

Media appliance 100 then post-produces the stored digitalrepresentation, using controller 108 and/or audio or video plugin storedin memory 110.

The post-produced digital representation is then stored 406 in memory110 (or in other storage medium such as optional on-appliance hard-diskor storage tape for storing data), displayed 407 on on-appliance displayunit 114, and/or sent for off-appliance display and/or exhibition (e.g.for IMAX display according to IMAX 15/70 format, or for TexasInstruments DLP (Digital Light Processing) format), or for digitalremastering according to IMAX's DMR (Digital Remastering) format, or forsatellite distribution (e.g. to Digital Audio Broadcast (DAB)distribution scheme to DAB enabled devices such as PDAs, cellularphones, personal audio and/or video players, or other devices forpresenting audio and/or video). Optionally, communication of mediaappliance 100 with other devices and/or services complies with ATSC DASE(Advanced Television Systems Committee Digital TV Application SoftwareEnvironment) architecture, incorporated herein by reference.

FIG. 5 is a flow diagram illustrating a method for optionally filling-ina template according to a preferred embodiment of the present invention.Starting 501 with a template, sense 502 a first scene according to thetemplate, and fill-in 503 sensed scene in template. If no additionalscene is desired 505, finish 506, else 504 proceed to step 502 andrepeat until done. Template is stored in memory 110 comprising suitableformat such as the Advanced Authoring Format (AAF).

FIG. 6 is a flow diagram illustrating a method for optionally taggingaudio and/or video representation with information contained in ameta-data structure. Upon sensing 601 a scene, the digitalrepresentation of the sensed scene is tagged 602 with meta-data.Meta-data comprises time, media appliance location (such as provided byGPS module 112), media appliance orientation and/or media applianceacceleration (such as provided by acceleration detector 113),multi-lingual features (allowing for translation, subtitles, voice-over,etc.), cues to a theater automation system (such as instructions forhouse lights to go up, half-way up, or down, or instructions to open orclose curtains, etc.), instructions for allowing or disallowing content(such as trailers or promotional clips) to play next to other similarcontent, information indicating suitability of content for differentaudiences such as children, information indicating any promotionaloffers, products and/or services (such as advertisements, productcatalogs and/or coupons for products and/or services), informationallowing for organizing and/or managing meta-data available toadvertisers and/or service providers, and/or other informationdescribing, identifying and/or relating to content. Tagging may be doneper scene, per frame, per audio and/or video stream (e.g. when multiplestreams are present), or per other defined segment of audio and/orvideo. For example, a video scene is tagged with meta-data comprising afield identifying the language used in the video scene. As anotherexample, a video stream is tagged with meta-data comprising a fieldindicating a warning against viewing by children.

FIG. 7 is a flow diagram illustrating a method for transferring dataand/or instructions from off-appliance source to on-appliance memory.After determining 701 off-appliance source, such as external repository(for templates, plugins, DRM data, encryption keys, media clips,security data, biometric data, GPS data, etc.), proceed by transferring702 data and/or instructions from determined off-appliance source toon-appliance memory 110.

In one embodiment, media appliance 100 is a member of a distributedgroup of media appliances 100, for example in a distributed network ofmedia appliances 100 and/or in a peer-to-peer configuration of mediaappliances 100. A media appliance 100 dynamically joins and/or leaves adistributed group of media appliances 100, in parallel and/or seriallywith other media appliances 100. Alternatively, media appliance 100initiates a distributed group of media appliances 100, allowing forother media appliance's 100 to dynamically join and/or leave the group.In one embodiment, the group of media appliances 100 collaborates tocover an event, such as a sporting event, a public political event (e.g.a rally), a family event (e.g. a wedding), or other event. Mediaappliances 100 tag sensed audio and/or video data as described above(e.g. with GPS information, time stamps, DRM meta-data, or otherinformation previously described), allowing reconstruction of coveredevent from the audio and/or video data collected by distributed mediaappliances 100. Memory 110 stores instructions and/or data forinitiating, joining, leaving and/or querying the status of orinformation about such a distributed group of media appliances 100.

Foregoing described embodiments of the invention are provided asillustrations and descriptions. They are not intended to limit theinvention to precise form described. In particular, it is contemplatedthat functional implementation of invention described herein may beimplemented equivalently in hardware, software, firmware, and/or otheravailable functional components or building blocks, and that networksmay be wired, wireless, or a combination of wired and wireless. Othervariations and embodiments are possible in light of above teachings, andit is thus intended that the scope of invention not be limited by thisDetailed Description, but rather by Claims following.

1. Media appliance comprising: means for sensing to generate an audio orvisual (A/V) signal; and reconfigurable logic or processor forprocessing the signal adaptively according to automatic recognition ofone or more audio or video feature extracted from the processed signal,whereby such extracted feature is usable for image or database query;wherein the signal is provided to a distributed group ofnetwork-extensible media appliances, such signal being provided usingprogrammable protocol translation via real-time or dynamic electronicsignal processing to convert a communication format of the providedsignal, at least one of the appliances in the distributed group therebypresenting electronically the signal via an audio or video renderer,whereby such distributed group of network-extensible media appliancescollaborates to reconstruct a covered event effectively by synchronizingprocessed A/V signals that are collected from distributed mediaappliances, such signals being synchronized to flow or branch accordingto a non-linear or non-deterministic script, template or storyboard. 2.The appliance of claim 1 wherein: said sensing means encodes the signalaccording to a video format enabling emotion recognition.
 3. Theappliance of claim 1 wherein: said reconfigurable logic or processorenables content, user or device authentication for securityintelligence.
 4. The appliance of claim 1 wherein: said reconfigurablelogic or processor tags the signal according to a digital rightsmanagement (DRM) flag or meta-datastructure.
 5. The appliance of claim 1further comprising: means for rendering the processed signal using anabstraction layer for presentation according to an ATSC format.
 6. Theappliance of claim 1 wherein: said reconfigurable logic or processorenables alternate rendering of the signal according to a non-linearbranching-point for a news, sports or movie program.
 7. The appliance ofclaim 1 wherein: said reconfigurable logic or processor processes thesignal synchronously with a delegated A/V stream in an appliance group.